Impact of the Covid-19 pandemic on the performance of machine learning algorithms for predicting perioperative mortality.

BACKGROUND: Machine-learning models are susceptible to external influences which can result in performance deterioration. The aim of our study was to elucidate the impact of a sudden shift in covariates, like the one caused by the Covid-19 pandemic, on model performance. METHODS: After ethical approval and registration in Clinical Trials (NCT04092933, initial release 17/09/2019), we developed different models for the prediction of perioperative mortality based on preoperative data: one for the pre-pandemic data period until March 2020, one including data before the pandemic and from the first wave until May 2020, and one that covers the complete period before and during the pandemic until October 2021. We applied XGBoost as well as a Deep Learning neural network (DL). Performance metrics of each model during the different pandemic phases were determined, and XGBoost models were analysed for changes in feature importance. RESULTS: XGBoost and DL provided similar performance on the pre-pandemic data with respect to area under receiver operating characteristic (AUROC, 0.951 vs. 0.942) and area under precision-recall curve (AUPR, 0.144 vs. 0.187). Validation in patient cohorts of the different pandemic waves showed high fluctuations in performance from both AUROC and AUPR for DL, whereas the XGBoost models seemed more stable. Change in variable frequencies with onset of the pandemic were visible in age, ASA score, and the higher proportion of emergency operations, among others. Age consistently showed the highest information gain. Models based on pre-pandemic data performed worse during the first pandemic wave (AUROC 0.914 for XGBoost and DL) whereas models augmented with data from the first wave lacked performance after the first wave (AUROC 0.907 for XGBoost and 0.747 for DL). The deterioration was also visible in AUPR, which worsened by over 50% in both XGBoost and DL in the first phase after re-training. CONCLUSIONS: A sudden shift in data impacts model performance. Re-training the model with updated data may cause degradation in predictive accuracy if the changes are only transient. Too early re-training should therefore be avoided, and close model surveillance is necessary.

Late versus early administration of convalescent plasma in hospitalized patients with covid-19: Results from the randomized open label clinical trial capsid

Background: Several observational studies suggested efficacy of COVID-19 convalescent plasma (CCP) but the results of several randomized clinical trials of CCP are not consistent. The trials differ in treatment schedules in terms of timing, volume and antibody content of CCP as well as enrolled patient populations and endpoints. The CAPSID was designed at the beginning of the pandemic and assessed the efficacy of neutralizing antibody containing high-dose COVID-19 convalescent plasma (CCP) in hospitalized patients with severe COVID-19. Methods: Patients (n=105) in 13 hospitals in Germany were randomized to either receive standard treatment and three units of CCP on days 1, 3 and 5 (total dose 846 ml) (n=53) or standard treatment alone (n=52). Patients in the control group with progress on day 14 could receive CCP (crossover group;n=7) on days 15, 17 and 19. The primary outcome was a dichotomous composite outcome of survival and no longer fulfilling criteria of severe COVID-19 on day 21. For Cross over patients a propensity matching with patients of the plasma group was performed. Results: Neutralizing antibodies were present at baseline in 18.2% of CCP and 19.2% of control group patients. In the ITT analysis the primary outcome occurred in 43.4% of patients in the CCP and 32.7% in the control group (p=0.32). The CCP group showed a trend for shorter times to clinical improvement (40 days, p=0.27) and discharge from hospital (20 days, p=0.24). Among those in the CCP group who received a higher or lower cumulative amount of neutralizing antibodies the primary outcome occurred in 56.0% and 32.1% of patients The high titer group showed significantly shorter intervals to clinical improvement or hospital discharge and a better overall survival (p=0.02). None of the patients in the crossover group (CG) achieved clinical improvement and survived. Comparing the CG to 14 CCP patients matched by baseline characteristics resulted in worse OS in the CG group (p=0.02) while comparison with 6 day 14 matched patients showed equal OS. Interpretation: CCP added to standard treatment did not result in a significant difference in the primary and secondary outcomes. A pre-defined subgroup analysis showed a signal of benefit for CCP among those who received a larger amount of neutralizing antibodies. A progress on day 14 is an indicator for poor outcome in COVID-19. Late administration of CCP is not supported by our results.

[Managing the pandemic-relocation concept for COVID-19 intensive care patients and non-COVID-19 intensive care patients in Baden-Württemberg]. / Die Pandemie bewältigen ­ Verlegungskonzept von COVID-19-Intensivpatienten und Non-COVID-19-Intensivpatienten in Baden-Württemberg.

BACKGROUND: A sharp rise in COVID-19 infections threatened to lead to a local overload of intensive care units in autumn 2020. To prevent this scenario a nationwide relocation concept was developed. METHODS: For the development of the concept publicly available infection rates of the leading infection authority in Germany were used. Within this concept six medical care regions (clusters) were designed around a center of maximum intensive care (ECMO option) based on the number of intensive care beds per 100,000 inhabitants. The concept describes the management structure including a structural chart, the individual tasks, the organization and the cluster assignment of the clinics. The transfers of intensive care patients within and between the clusters were recorded from 11 December 2020 to 31 January 2021. RESULT: In Germany and Baden-Württemberg, 1.5% of patients newly infected with SARS-CoV­2 required intensive care treatment in mid-December 2020. With a 7-day incidence of 192 new infections in Germany, the hospitalization rate was 10% and 28-35% of the intensive care beds were occupied by COVID-19 patients. Only 16.8% of the intensive care beds were still available, in contrast to 35% in June 2020. The developed relocation concept has been in use in Baden-Württemberg starting from 10 December 2020. From then until 7 February 2021, a median of 24 ± 5/54 intensive care patients were transferred within the individual clusters, in total 154 intensive care patients. Between the clusters, a minimum of 1 and a maximum of 15 (median 12.5) patients were transferred, 21 intensive care patients were transferred to other federal states and 21 intensive care patients were admitted from these states. The total number of intensive care patients transferred was 261. CONCLUSION: If the number of infections with SARS-CoV­2 increases, a nationwide relocation concept for COVID-19 intensive care patients and non-COVID-19 intensive care patients should be installed at an early stage in order not to overwhelm the capacities of hospitals. Supply regions around a leading clinic with maximum intensive care options are to be defined with a central management that organizes the necessary relocations in cooperation with regional and superregional rescue service control centers. With this concept and the intensive care transports carried out, it was possible to effectively prevent the overload of individual clinics with COVID-19 patients in Baden-Württemberg. Due to that an almost unchanged number of patients requiring regular intensive care could be treated.